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intkouik: i ion 

Our  studies  of  learning  have  several  major  foci.  First,  how  is 
ini ormat ion  stored  in  the  human  memory  system:  how  is  new  information  in- 
tegrated with  the  existing  knowledge/  Second,  what  are  the  mechanisms 
used  by  the  learner  to  select  information  from  the  learning  environment 
and  integrate  it  with  his  existing  knowledge  structure?  Third,  how  doe 
the  tutor  model  the  Knowledge  state  of  the  student  and  use  that  model  to 
guide  the  tutorial  dialog? 

We  have  recently  developed  the  Fl.OW  system,  a multipurpose  facility 
for  studies  of  the  learning  process.  Some  of  the  work  with  the  FLOW  system 
has  been  described  previously  (Norman,  Centner  &.  Stevens,  1974;  Norman, 
Centner  Stevens,  in  press).  This  report  will  be  concerned  primarily 
with  the  development  of  an  automated  tutor  which  con:  tructs  a simple  model 
of  the  student  and  uses  that  model  to  assist  the  student  when  needed. 

Overview  of  tile  svstem 

FLOW  is  a simple,  interactive  computer  language  whose  commands  are 
designed  for  string  manipulation.  Although  FLOW  is  not  intended  for  prac- 
tical computer  applications,  it  shares  many  important  properties  with  com- 
mon computer  languages  and  is  particularly  suitable  for  our  studies  in 
learning.  The  section  on  the  FLOW  languagt  describes  the  language  in  detail. 

The  FLOW  system  is  based  around  a minicomputer.  A cathode  ray  tube 
(CRT)  display  terminal  for  the  student  is  attached  to  the  minicomputer.  The 
experimenter,  who  is  normally  in  another  room,  can  monitor  the  student's 


terminal  on  a television  display.  The  minicomputer  is  connected  to  a large 
computer  in  which  the  automated  tutor  program  resides.  The  experimenter 
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h;is  a terminal  connected  to  the  large  computer  which  he  can  use  to  inter- 
act with  the  automated  tutor  program  and,  through  that  program,  wi  tli  the 
student.  There  are  also  facilities  for  recording  complete  protocols  of 
the  experimental  session.  llie  equipment  used  in  the  l'LOl\  s\  stem  is  dcs 

cribed  in  detail  in  the  section  on  Hardware. 

Over  the  past  few  y^ars,  a number  of  different  approaches  have 
been  taken  towards  the  development  of  computer-based  instructional  systems. 
Hiese  systems  can  he  generally  classed  as  one  of  three  types.  The  first 
and  most  common  type  is  based  on  the  sequence:  present  text,  tost,  and  branch 
The  student  is  given  a section  of  text  material  and  then  tested  on  that 
material, usually  with  a multiple  choice  test.  Depending  on  the  result  of 
the  test,  the  student  can  either  be  given  the  next  text  in  sequence,  or  be 
sent  back  to  a previous  section,  or  be  side-tracked  to  a remedial  section. 

The  second  type  of  system  is  often  called  a generative  system.  The  SCHOLAR 
system  originally  developed  by  Carbonell  (1970)  is  a fine  example  of  a 
generative  system.  Here  the  information  (texts,  questions  and  answers)  are 
not  explicitly  stored,  but  instead  there  is  a database  in  which  information 
about  the  subject  matter  to  be  taught  is  stored  as  a semantic  network.  Gen- 
erative routines  in  SCHOLAR  can  a cess  the  database  to  prepare  text  or  ques- 
tions for  the  student  or  to  answer  questions  from  the  student.  A third  type 
of  computer-based  instructional  system  is  the  simulation  system.  In  this 
tvpe  the  computer  is  programmed  to  simulate  some  domain  of  interest,  such 
as  a regulated  power  supply  (Brown,  Burton  & Bell,  1974)  The  student  has 
a number  of  operations  he  can  perform  on  the  simulation  and  learns  by  see- 
ing the  results  of  these  operations. 
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Actually,  Brown's  ystem  can  also  carry  on  sim|)le  diatoms  wi  th  the 
student.  Our  H.OW  system  takes  a similar  approach,  giving  the  student  an 
interactive  domain  to  experiment  with  (an  implementation  of  the  FLOW 
language)  and  tutorial  assistance.  (For  examples  of  related  systems  see 
Barr,  Beard  Atkinson,  1974;  Koffman,  Blount,  Gilkey,  Perry  & Wei,  1 97  3 ; 
Goldberg,  1 97  3.)  1’he  automated  tutor  in  the  FLOW  system,  however,  differ 
considerably  from  previous  systems.  In  some  preliminary  experiments,  the 
student  learned  FLOW  primarily  from  written  instructions,  entered  his  pro- 
grams into  the  computer  and  modified  his  programs  based  on  the  results. 
Meanwhile  a human  tutor  was  watching  over  the  shoulder  of  the  student 
and  would  answer  questions  or  interrupt  if  the  student  appeared  to  be 
having  difficulty.  The  automated  tutor  is  meant  to  duplicate  the  func- 
tion of  that  human  tutor.  The  student  still  learns  primarily  from  written 
instructions,  but  the  autotutor  tries  to  follow  his  progress  and  keep 
track  of  where  he  is  in  the  instructions  by  monitoring  what  he  types  into 
the  computer.  The  student  need  not  follow  the  written  instructions  exact- 
ly, but  may  move  back  or  skip  ahead.  The  autotutor  will  try  to  follow  and 
be  ready  to  give  appropriate  help  if  the  student  gets  in  trouble. 
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THi:  FLOW  1ANGUAGE 

Origins 

The  FlX)W  language  was  originally  developed  by  Professor  .Jeffrey 
Raskin  of  the  Visual  Arts  Department  at  L1CSD.  He  wanted  to  teach  computer 
programming  to  humanities'  students  who  often  either  had  little  experience 
with,  or  actively  disliked  mathematics  and  computers.  FLOW  was  conceived 
as  an  introductory  computer  language  that  would  be  non-threatening  and 
easy  to  understand  while  still  possessing  the  basic  functions  and  construc- 
tions of  typical  computer  languages.  After  first  learning  FLOW,  the  stu- 
dent would  then  go  on  to  a standard  language  such  as  BASIC  (Raskin,  in  press). 

FLOW 

Table  1 shows  the  commands  and  statements  in  the  implementation  of 
FLOW  used  in  these  experiments.  (After  the  experiments  described  in  this 
report,  we  renamed  most  of  the  FLOW  commands  and  statements  to  make  them 
more  mnemonic.  See  the  section  on  Subsequent  Developments.) 

In  our  implementation  of  FLOW,  the  student's  terminal  is  connected 
tc  the  minicomputer  via  a full-duplex  link.  This  means  that  when  the  stu- 
dent presses  a key  on  his  terminal,  the  character  is  first  sent  to  the  mini- 
computer, examined  by  the  computer  and  then  echoed  back  for  display  on  the 
terminal.  This  arrangement  allows  two  interesting  features.  The  first 
feature  is  called  "typing  amplification"  by  Raskin.  As  soon  as  the  mini- 
computer can  unambiguously  recognize  a command  or  statement  from  the  char- 
acters input  by  the  student  it  supplies  the  remaining  letters  of  that 
command.  Since  most  FLOW  commands  begin  with  different  letters,  it  is 


-5- 


l 

I 

l 

I 

I 

I 

I 

8 

I 

I 

I 

I 


usually  necessary  to  type  in  only  the  first  letter  of  a statement.  With 
che  commands  and  statements  listed  in  Table  1,  only  the  underlined  letters 
need  to  lie  typed  in,  the  computer  will  supply  the  rest.  Second,  since  the 
computer  can  examine  each  character  before  displaying  it  on  the  terminal, 
syntactically  incorrect  characters  can  be  rejected.  Together  these  two  lea- 
tures  virtually  eliminate  typing  and  syntactic  errors,  two  major  sources  of 
frustration  for  beginning  programmers.  To  illustrate  how  they  work,  suppose 
a student  is  entering  his  program.  After  completing  a line,  the  computer 
does  a carriage  return  and  line  feed,  provides  a new  line  number  and  then  waits 
for  a student  input.  At  this  point  the  minicomputer  will  accent  anv  letter 
which  begins  a FLOW  command  or  statement  or  a number  (indicating  that  the  stu- 
dent wants  to  change  the  line  number).  If  the  student  presses  the  "P"  key 
the  word  "PRINT"  is  uisplayed  on  the  terminal  and  the  minicomputer  waits  for 
a legal  completion  of  a PRINT  statement.  (The  legal  characters  at  this  point 
are  I,  ',  and  R) . If  the  student  then  presses  "J",  the  minicomputer  will 
momentarily  display  a "J"  with  an  audible  lone  then  backspace,  erase  the  "J" 
and  wait  for  a legal  character.  An  "I"  will  be  accepted  as  a legal  character, 
the  minicomputer  displays  "IT"  dots  a carriage  return,  linefeed,  displays  a new 
line  number  and  waits  for  the  next  statement.  It  is  not  a recommended  pro- 
gramming practice,  but  as  a graphic  demonstration  of  these  two  features,  it 
is  possible  to  bang  away  at  the  keyboard,  and  always  produce  syntactical Iv 
correct,  executable  FLOW  programs,  although,  of  course,  there  is  no  guarantee 


of  what  they  will  do. 


Advantages  of  FLOW  lor  I. earning  Studies 


FLOW  has  a number  of  advantages  as  a subject  matter  foy  use  in  stud- 
ies of  learning.  Along  with  other  computer  languages,  F1X)W  involves  both 
conceptual  and  procedural  knowledge.  It  is  problem  oriented,!  There  are  a 
large  variety  of  problems  that  can  be  posed  to  exercise  the  student's 
developing  knowledge.  The  language  itself  and  the  tasks  form  a small,  concise, 
and  well  defined  body  of  knowledge.  The  statement  of  a problem  and  whet  con- 
stitutes an  acceptable  solution  ate  normally  fairly  clear,  although  there  may 
be  some  debate  about  the  quality  of  a particular  solution.  The  students  in 
most  of  our  studies  were  university  undergraduates,  and  it  is  relatively  easy 
to  find  subjects  from  that  group  with  little  or  no  experience  with  computer 
1 anguages . 

FLOW  also  has  some  unique  advantages  as  a subject  matter.  With  typing 
amplification  and  the  rejection  of  syntactically  incorrect  characters,  two 
major  sources  of  uninteresting  errors  are  eliminated.  The  errors  we  are  left 
with  are  concerned  with  the  meaning  of  individual  statements  and  how  groups 
of  statements  interact.  Since  FLOW  is  quite  a simple  language,  a lot  of  in- 
teresting learning  takes  place  within  the  first  hour.  The  subject  matter 
is  complex  enough,  however,  to  require  up  to  10  hours  to  master.  This  forms 
a convenient  time  span  for  experimental  studies.  Subjects  usually  enjoy 
learning  FLOW  with  its  interactive  nature  and  low  level  of  frustration, 
and  are  highly  motivated  to  learn. 

The  last  advantage  of  FLOW  derives  from  the  fact  that  it  is  primarily 
a computer  mediated  task.  This  leads  naturally  to  the  possibility  for 
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computer  interaction  during  the  instructional  sequence  and  to  _*a se  of  collec- 
tion of  experimental  data.  Also  since  the  task  is  primarily  verbal,  it  is 
easy  to  collect  detailed  protocols  of  the  tutorial  interaction  and  attempts 
at  problem  solving  tor  later  analysis. 
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ilARDWARE 

Although  this  report  is  concerned  mainly  with  the  automated  tutor, 
the  FLOW  system  is  designed  for  a number  of  different  types  o’  experimental 
studies.  Figure  1 is  a schematic  diagram  of  our  equipment.  The  equipment 
(except  for  the  B6700)  is  located  in  two  adjoining  experimental  booths. 

The  system  is  based  around  a Microdata  810  minicomputer  with  24K 
bytes  of  memory  (the  applications  described  here  require  only  8K  bytes). 

The  student  works  at  a Scientific  Measuring  Systems  1440  CRT  terminal  con- 
nected to  the  minicomputer  via  a full-duplex  1200  baud  (120  characters 
per  second)  link.  A video  output  from  the  CRT  terminal  is  connected 
to  a video  monitor  in  the  experimenter's  booth  duplicating  the  display 
on  the  student's  terminal.  The  signal  going  from  the  minicomputer  to 
the  student's  terminal  which  contains  all  of  the  information  to  be  dis- 
played on  the  terminal  including  vertical  and  horizontal  spacing  also 
goes  to  a recording  unit.  This  consists  of  a modem  converting  the  dig- 
ital signal  produced  by  the  minicomputer  into  a sequence  of  tones  which 
are  then  recorded  on  an  audio  tape  recorder.  The  tape  recorder  can 
record  up  to  four  tracks  of  information  such  as  verbal  comments  from 
the  student  and  tutor  in  addition  to  the  terminal  display.  Later  when 
the  tape  recording  is  played  back  through  the  modem,  the  tones  are  con- 
verted back  into  digital  signals  which  faithfully  duplicate  the  experi- 
mental session  on  the  CRT  terminal. 

The  automated  tutor  part  of  our  system  is  contained  in  programs 
residing  on  the  UCSD's  Burroughs  6700  computer,  which  is  connected  to 
our  minicomputer  via  two  half-duplex  4800  baud  lines.  (The  two  half- 
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duplex  lint's  can  bo  replaced  with  a single  full-duplex  connection  when 
the  system  will  support  full  duplex.)  The  B6700  computer  is  also  con- 
nected to  a Tektronix  4010  di-  play  terminal  in  the  experimenter's  booth 
which  is  used  to  program  the  B6700  and  to  communicate  between  experimen- 
ter and  the  automated  tutor  during  tutorial  sessions.  In  addition  the 
B6700  prepares  a protocol  of  the  tutorial  session  for  later  analysis. 

The  experimenter's  booth  also  contains  a keyboard  and  Teletype  terminal 
which  are  used  to  communicate  with  the  minicomputer. 

The  minicomputer  controls  most  of  the  flow  of  information.  It 
monitors  the  siudent's  keyboard,  interprets,  and  runs  his  programs, 
and  writes  on  the  student's  display  screen.  Input  from  the  student  is 
summarized  and  sent  to  the  automated  tutor  on  the  B6700  and  all  mes- 
sages from  the  autotutor  are  relayed  through  the  minicomputer.  The 
major  information  link  outside  the  minicomputer  is  that  the  experimenter 
can  interact  directly  with  the  autotutor. 
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THE  AUTOMATED  TUTOR 

The  Printed  Instructions 

We  decided  to  base  the  teaching  of  FLOW  on  a set  of  printed  ir  - 
structions  primarily  because  of  ease  of  preparation,  flexibility,  and 
convenience  of  use  for  the  student.  Few  new  educational  technologies 
can  rival  the  printed  page  which  is  self-paced,  portable,  easily  search- 
ed and  inexpensive.  Figure  2 shows  a typical  portion  of  the  instructions. 

The  instructions  are  divided  into  seven  units,  each  of  which  in- 
volves three  types  of  tasks:  reading,  entering  practice  programs,  and 
solving  problems.  These  three  types  of  tasks  can  be  seen  in  Figure  2. 

The  first  part  of  the  figure  is  explanatory  text.  Then  there  is  a short 
program  using  the  new  statement  which  the  student  can  enter  and  run. 
Finally  the  student  is  given  a programming  problem  to  solve.  All  the 
units  have  this  structure;  there  are  one  or  more  sections  of  text  and 
practice  programs,  and  then  the  student  is  given  a problem  to  solve  at 
the  end  of  the  unit.  We  ask  the  student  to  not  go  on  to  the  next  sec- 
tion until  he  is  told  to.  In  fact  the  autotutor  will  send  him  on  as 
soon  as  he  solves  the  problem.  The  student  can  go  back  in  the  instruc- 
tions whenever  he  likes  but  we  thought  he  should  not  move  on  to  the 
next  unit  until  he  had  mastered  the  current  unit.  We  were  also  afraid 
the  autotutor  might  have  difficulty  if  it  thought  the  student  was  work- 
ing on  Problem  4 when  he  was  actually  working  on  Problem  5. 

New  commands  and  statements  are  introduced  in  each  section.  With 
typing  amplification  naive  students  often  accidentally  enter  statements 
which  they  know  nothing  about,  leading  to  general  confusion.  To  avoid 
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this  problem,  the  minicomputer  initially  does  not  respond  to  any  key 
Iron  the  student's  keyboard.  As  the  student  moves  through  the  in- 
strut  tions,  keys  are  activated  at  the  time  that  the  corresponding  state- 
ments are  introduced.  While  the  student  is  working  on  Unit  1 , for 
instance,  only  the  H,P,',R,L,N  and  E keys  are  activated  (corresponding 
to  HELP,  PRINT  'string',  RUN,  LIST  and  NEW).  When  the  student  success 
fully  completes  the  problem  at  the  end  of  unit  1,  the  autotutor  gives 
him  a congratulatory  message,  tells  him  to  go  on  to  Unit  2,  and  ac- 
tivates the  keys  needed  for  that  unit. 


Following  the  Student 

During  the  development  of  the  autotutor,  all  its  functions  were 
originally  carried  out  by  a human  tutor.  Parts  of  the  tutorial  func- 
tion were  then  taken  over  by  the  computer  until  it  was  completely  auto- 
mated. The  human  tutor  typically  forms  a rather  simple  model  of  the  stu- 
dent containing  the  following  information: 

1)  where  the  student  is  in  the  instructions,  and  2)  his  current  program. 
The  human  tutor  refers  to  the  instructions  and  general  knowledge  to 
generate:  3)  the  student's  next  expected  input,  and  4)  the  maximum  time 
it  should  take  him  to  enter  the  input.  If  the  student  entered  the  expect- 
ed input,  the  human  tutor  would  advance  the  student  model  to  the  next 
point  in  the  instructions.  If  the  student  entered  an  unexpected  input, 
the  tutor  would  use  the  instructions  and  general  world  knowledge  to  try 
and  deduce  why  the  student  had  done  that,  and  where  he  now  was  in  the 
instructions.  Rather  than  have  the  autotutor  generate  maximum  pause 
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lengths  and  the  meaning  of  unexpected  inputs  from  the  instructions 
as  a human  tutor  does,  we  decided  that  that  would  be  too  difficult  a 
task  and  entered  the  maximum  pause  length  and  a list  of  inputs  explic- 
itly for  each  state. 

One  of  the  major  tasks  of  the  autotutor  is  to  follow  the  stu- 
dent. The  student  normally  proceeds  sequentially  through  the  instruc- 
tions. But  from  time  to  time  he  may  make  errors  and  decide  to  repeat 
things  or  even  go  back  to  an  earlier  part  of  the  instructions.  A con- 
fident student  may  skip  over  some  of  the  practice  programs  and  start 
immediately  on  the  problem.  Through  all  this,  the  autotutor  monitors 
what  the  student  is  typing  into  his  terminal  and  tries  to  keep  track 
of  where  he  is  in  the  instructions. 

The  instructional  sequence  is  divided  into  136  states.  During 
the  practice  parts  of  the  instructions,  a state  usually  corresponds  to  a 
FLOW  statement  or  command,  although  in  the  first  unit,  states  may  cor- 
respond to  individual  characters  and  partial  commands.  During  the  prob- 
lem part  of  the  instructions,  each  problem  corresponds  to  essentially 
a single  state.  Each  state  has  associated  with  it  a list  of  information 
which  the  autot  ;tor  uses  to  follow  the  student.  These  states  and  their 
associated  li  is  are  all  that  the  autotutor  knows  about  the  instruc- 
tions, Because  the  autotutor  employs  two  distinct  strategies  for  the 
practice  states  and  the  problem  states,  we  will  discuss  each  of  these 
separately. 

Practice 

The  autotutor  gives  three  types  of  messages  during  the  practice 
part  of  a unit:  1)  standard  messages  in  response  to  a request  for  help, 

or  if  the  student  pauses  too  long,  based  on  the  next  expected  input 
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(Try  typing  a_ );  2)  messages  in  response  to  a serious  error, 

(usually  a NEW  out  of  sequence)  which  tell  what  should  have  been  typed 
and  send  the  student  back  to  a previous  section  to  try  again  (You  should 

have  typed  ; please  go  back  to  Section  );  3)  special  message;- 

peculiar  to  a given  state  or  class  of  states  which  clarify  usage  of  cer- 
tain keys  or  commands  or  explain  procedures  (e.g.,  If  during  the  first 
unit  the  student  types  the  letter  "C"  when  he  should  have  typed  the  num- 
ber "0",  he  gets  the  message,  "Zero  (0)  is  located  to  the  right  of  9 
Don't  use  the  letter  0 for  zero.") 

Io  illustrate  how  the  autotutor  works  when  the  student  is  in  the 
practice  part  of  the  instructions,  let's  look  in  detail  at  one  section 
of  the  instructions  in  Figure  2.  In  this  unit  we  introduce  the  PRINT 
RETURN  statement.  In  the  practice  part  <;e  give  a brief  explanation  of 
the  statement  and  then  a practice  program  using  the  new  statement  which 
the  student  is  expected  to  enter  and  run. 

When  the  student  successfully  completes  the  previous  problem 
(Problem  2)  the  autotutor's  model  of  the  student  enters  State  276.  (The 
states  are  not  all  numbered  consecutively.)  When  the  student  then  enters 
a NEW  command,  the  minicomputer  erases  the  screen  and  gives  the  student 
a line  number  (010)  for  his  new  program.  Meanwhile  the  autotutor  advances 
the  student  model  to  State  277.  We  will  examine  State  277  in  detail. 

Associated  with  State  277  are  four  possible  student  inputs  and  a 
maximum  pause  length.  The  autotutor  thinks  that  the  student  is  about 
to  type  in  the  practice  program  and  waits  for  a student  input.  The  five 
| inputs  which  it  is  looking  for  in  particular  are: 

l 


14- 


PRIN1  'HELLO' 


PRINT  'BASKET' 


PRINT  'anything  else' 


This  is  what  the  autotutor  expected  and  it  simply 
advances  the  student  model  to  the  next  state 
(State  278). 

This  is  the  first  statement  for  the  solution  of 
Problem  2,  so  the  autotutor  assumes  that  the  stu- 
dent has  skipped  the  practice  section  and  advances 
the  student  model  to  the  beginning  of  Problem  2 
(State  282). 

The  autotutor  assumes  that  it  just  has  a cieative 
student  who  didn't  like  the  word  HELLO  and  advanc- 
es him  to  State  278. 


NEW 


The  autotutor  assumes  that  the  student  had  made 
some  errors  and  decided  to  begin  again.  The  stu- 
dent model  is  kept  in  State  277,  and  to  keep  some 
control  over  the  chaos,  the  student  is  asked  to 
start  at  Section  13. 


HELP 


The  autotutor  sends  the  message:  TRY  TYPING  A 

PRINT  'HELLO'.  The  student  model  remains  in 
State  277. 


Any  other  input  is  assumed  to  be  an  error  which  the  student  will  notice  and 
correct.  The  student  model  is  left  in  State  277. 


J 


The  maximum  pause  length  for  State  277  is  30  seconds.  Failure  to  press  any 
key  for  30  seconds  is  equivalent  to  typing  HELP.  The  student  gets  the  mes- 
sage TRli  TYPING  A PRINT  'HELLO'"  and  the  student  model  remains  in  State  277 
(The  maximum  pause  lengths,  which  vary  from  30  seconds  to  5 minutes  are  en- 
tered explicitly  by  the  experimenter.) 


Problems 

When  the  student  fin'shes  the  practice  part  of  the  instructions,  the 
student  model  is  advanced  to  the  state  corresponding  to  the  next  problem. 

A problem  essentially  corresponds  to  a single  state;  running  a correct  solu- 
tion to  the  problem  advances  the  student  to  the  next  state.  Associated 
with  each  problem  state,  the  autotutor  has  a program  which  is  a correct 
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solution  ol  the  problem.  11  the  program  requires  input  (in  FLOW,  th.it 
would  be  in  the  form  oi  a TEXT  statement),  the  airotutor  will  also  have 
several  test  inputs.  As  the  student  types  in  his  program,  tin  minicomput- 
er sends  the  statements  to  the  autotutor. 

II  the  student  runs  his  program,  asks  for  help  or  pauses  t jo  long, 
the  autotutor  runs  the  student's  program  along  with  the  coriect  program, 
substituting  in  its  test  inputs  if  necessary.  If  the  two  programs  gen- 
erate essentially  the  same  output,  the  student's  program  is  judged  correct, 
and  the  student  is  given  a congratulatory  message  and  sent  on  to  the  next 
unit.  The  autotutor  is  somewhat  lenient  about  certain  discrepancies  in 
the  student's  output  such  as  extra  spaces.  Within  the  lather  small  scope 
of  FLOW  and  the  programming  problems  we  use,  our  procedure  of  comparing 
the  outputs  of  a correct  program,  and  the  student's  program  works  quite  well. 
If  the  outputs  are  not  essentially  the  same,  the  autotutor  does  a line 
by  line  comparison  ot  the  programs  and  informs  the  student  of  the  first 
discrepancy  it  finds  between  the  student's  program  and  the  correct  program. 

The  program  is  viewed  topologically  as  groups  of  non-control  state- 
ments bounded  by  control  statements.  (Control  statements  alter  the  linear 
flow  of  command  ol  execution;  thev  are  STOP,  JUMP  TO  nnn,  and  IF  IT  IS  ' c ' 
JUMP  TO  nnn.)  The  non-control  parts  are  examined  statement  by  statement 
as  previously  described.  When  a control  statement  is  encountered, 
the  branch  is  taken  to  the  new  portion  of  the  program  and  this  is  examin- 
ed. This  is  a recursive  process,  and  if  this  new  portion  pa-ses  inspec- 
tion, thin  the  statement  following  the  initial  control  stat  r..  nt  is  examined. 
Thus  the  whole  executable  program  will  be  canvassed  fo-  errors. 
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Kxnmplc  of  a Tutorial  Interaction 

To  illustrate  the  action  o:  the  auuotutor,  we  will  present  a pm - 
tion  of  a tutorial  interaction  along  with  the  autotutor's  interpretation 
of  t*lc  situation  in  terms  of  its  modification  of  the  student  model  . Al- 
though this  dialog  is  taken  from  actual  tutorials,  we  have  blended  to- 
gether several  tutorials  with  different  students  to  be  able  to  show  a 
variety  of  student  errors  and  how  the  autotutor  handles  them.  As  we 
start  this  excerpt,  the  student  has  just  correctly  solved  Problem  2. 
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Summary  of  the  Automated  Tutor 

As  tlu>  student  works  1 rom  written  instructions,  tlic  autotutor  mon- 
itors tlu  student's  keyboard  inputs  and  tries  to  follow  the  stu- 
dent's progress  through  a series  of  states  corresponding  to  the  written 
instructions.  The  states  corresponding  to  the  practice  parts  of  tie  in- 
structions have  an  expected  pause  length  and  a list  of  possible  i.puts 
associated  with  them.  Corresponding  to  each  possible  input  is  a destina- 
tion state  and  possibly  a message.  In  a given  state,  if  the  student 
types  in  one  of  the  inputs  on  the  list,  the  autotutor  prints  out  the 
message  if  any  and  advances  the  model  of  the  student  to  the  destination 
state.  If  the  student  types  in  an  input  not  on  the  list,  the  student 
model  remains  in  the  ,amc  state.  Except  for  these  messages,  which  are 
rare,  the  only  time  that  the  student  is  aware  of  the  presence  of  the 
autotutor  is  when  he  asks  tor  help  or  pauses  for  a long  time  and  exceeds 

the  expected  pause  length.  Then  the  autotutor  prompts  the  student  with 
the  next  expected  input. 

Somewhat  surprisingly  it  has  been  our  experience  that  the  autotutor 
usually  makes  fewer  errors  in  following  the  student  through  the  practice 
parts  than  a human  tutor.  Analysis  of  protocols  from  early  sessions  with 
human  tutors  showed  that  the  human  tutors  often  were  quite  mistaken  in 
their  judgments  of  where  the  student  was  in  the  instructions.  Hie  main 
problem  seemed  to  be  that  crucial  key  presses  were  missed  due  to  lack 
ot  attention.  Of  course,  eternal  vigilance  is  one  of  tie  supreme  virtues 
of  a computer,  and  this  seems  to  give  it  the  edge  in  this  task. 
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Thi  ope  r«i  t i on  of  the  autotutor  while  the  student  is  wor  ung  on  a 
problem  is  much  dillermt.  Here  the  autotutor  only  has  an  example  of  a 
correct  solution  to  the  programming  prnb.em.  W i 1 1 1 tlie  exception  of  two 
messages  which  are  sometimes  given  early  in  the  problem,  all  of  the  mes- 
sages from  the  autotutor  are  generated  algorithmically  based  o t a compar 
ison  of  tlu  student's  program  and  the  correct  program.  In  addition  to 
assisting  the  student  with  an  inc  'rrect  program,  the  autotutoj  also  runs 
the  student's  program  whenever  the  student  runs  it,  and  if  th  :*  program 
seems  correct,  the  autotutor  congratulates  the  student  and  asks  him  to 
start  on  the  next  unit. 
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Sl'B  SEQUENT  DEVELOPMENTS 

Command  Names 

Students  learning  FLOW  often  found  some  of  the  names  of  FLOW 
statements  and  commands  obscure  oj  contusing.  In  an  effort  to  clarify 
their  functions,  wt  have  renamed  many  of  the  instructions.  Table  2 
lists  the  old  instructions  and  the  corresponding  new  instruction. 

We  have  also  altered  the  display  generated  by  the  WALK  command 
to  make  it  clearer  what  is  happening:  eight  lines  of  the  student's  pro- 
gram are  displayed  on  the  upper  portion  of  the  screen  along  with  a 
pointer  indicating  the  statement  currently  being  executed. 

Conceptual  FLOW 

One  of  our  original  interests  was  to  teach  students  the  FLOW 
commands  and  statements  and  study  the  spontaneous  development  of  progran- 
m'.ng  concepts.  Thus  the  written  instructions  and  automated  tutor  des- 
cribed in  this  report  are  oriented  around  che  statements  in  the  FLOW 
language.  Concepts  such  as  control  transfer  and  loops  are  not  explicitly 
treated.  Some  tutorials  we  have  conducted  indicate  that  instruction  bas- 
ed on  programming  concepts  rather  than  the  specific  statements  would  be 
more  fruitful,  especially  for  students  with  no  previous  experience  with 
computers  or  programming.  To  produce  a FLOW  instructional  system  similar 
to  the  one  discussed  here  but  with  a conceptual  emphasis  would  involve 
rewriting  the  instructions  and  modifying  the  autotutor  to  analyze  student 
programs  in  terms  of  the  relevant  concepts.  An  interest  of  ours,  in 
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another  part  of  the  FLOW  study,  is  t li e manner  in  which  students  form  their 
developing  knowledge  into  schemata.  These  schemata  usually  seem  to  be  con- 
cerned with  concepts  such  as  loops.  A version  of  instruction  in  FLOW 
oriented  towards  the  programming  concepts  would  thus  also  connect  in  more 
directly  with  our  studies  of  schemata  and  bow  they  are  used  in  learning. 
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Commands 

HELP 

NEW 

RUN 

WALK 


LIST 

Program  Statements 
PRINT  'character  string1 
PRINT  RETURN 
PRINT  IT 
JUMP  TO  nnn 

TEXT  IS  'string' 

GET  IT 

IF  IT  IS  'x'  JUMP  TO  nnn 

COMMENT  character  string 
MAKE  COUNTER  ZERO 
ADD  ONE  TO  COUNTER 
DECREASE  COUNTER  BY  ONE 
PRINT  COUNTER 


TABLE  1 

FLOW  INSTRUCTIONS 


Requests  help  from  tutor  or  autotutor. 

Erases  current  program. 

Runs  current  program. 

Steps  through  the  current  program.  One  line  of  the  pro- 
gram is  executed  each  time  the  space  bar  is  pressed. 

The  program  statements  are  displayed  as  they  are  execut- 
ed on  the  upper  part  of  the  screen,  variables  are  dis- 
played in  the  middle  of  the  screen,  and  any  resulting 
output  is  displayed  in  the  lower  part  of  the  screen. 

This  command  is  a very  useful  debugging  aio. 

Displays  the  current  program. 


Displays  the  character  string. 

Does  a carriage  return  and  line  feed. 

Displays  the  variable  IT. 

Statement  number  nnn  is  executed  next.  (If  there  is  no 
statement  with  line  number  nnn,  the  statement  with  the 
next  highest  line  number  is  executed.  If  all  the  line 
numbers  are  lower  than  nnn,  the  program  stops.) 

Defines  a character  string  to  be  used  as  data. 

Sets  the  variable  IT  to  successive  characters  in  the  TEXT 
statement . 

If  the  variable  IT  is  the  same  as  the  character  in  quotes, 
the  JUMP  is  performed. 

No  effect. 


Displays  the  variable  COUNTER. 


IF  COUNTER  IS  tddcid  JUMP  TO  nnn 


I 

i 

t 

f 

Notes: 

f 

I 

i 

? 

f 

f 

! 

f 

! 

! 

1 

! 

! 

1 

! 

I 


V> 


TABLE  1 (cont'd.) 


System  Commands  are  executed  immediately  on  entry.  Program  statements  are 
used  to  write  stored  programs. 

The  COMMENT  statement  and  the  ones  below  it  were  not  used  in  the  studies  des- 
cribed in  this  report. 

The  programmer  types  in  only  the  underlined  characters,  the  computer  supplies 
the  rest.  (See  the  section  on  Typing  Amplification.) 

The  programmer  must  type  in  the  first  two  letters  of  the  NEW  command.  This 
is  designed  to  help  prevent  accidental  erasure  of  programs. 

lho  lower  case  letters  in  the  statements  correspond  to  slots  to  be  filled  in 
by  the  programmer.  In  particular,  the  phrases  "character  string"  or  "string" 
can  be  replaced  by  any  string  of  characters  and  the  phrases  "dddd"  and  "nnn" 
are  meant  to  be  replaced  by  numbers  of  up  to  1 and  .1  digits  respectively. 
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' 

NEW 

Old  Instruction 
HELP 
NEW 
RUN 
WALK 
LIST 

COMMENT  string 
PRINT  1 string1 
PRINT  IT 
PRINT  COUNTER 
PRINT  RETURN 
JUMP  TO  nnn 

1 TEXT  IS  ' string' 

GET  IT 

IF  IT  IS  'x'  JUMP  TO  nnn 
STOP 

MAKE  COUNTER  ZERO 

♦ ADD  ONE  TO  COUNTER 

DECREASE  COUNTER  BY  ONE 
IF  COUNTER  IS  tdddd  JUMP  TO 


TABLE  2 

iW  INSTRUCTIONS 

New  Instruction 

HELP 

ERASE 

RUN 

WALK 

LIST 

COMMENT  string 
DISPLAY  "string" 

DISPLAY  VARIABLE 
DISPLAY  COUNTER 
BEGIN  ON  NEXT  LINE 
JUMP  TO  nnn 
TEXT  IS  "string" 

GET  VARIABLE 

IF  VARIABLE  IS  "x"  JUMP  TO  nnn 
QUIT 

MAKE  COUNTER  ZERO 
ADD  ONE  TO  COUNTER 
SUBTRACT  ONE  FROM  COUNTER 
IF  COUNTER  IS  -ddd  JUMP  TO  nnn 


nnn 


Figure  1 

The  FLOW  Experimental  Facility 


PROBLEM  5: 


Section  12 


You  are  now  going  to  learn  how  to  make  the  computer  type  a vertical 
string  of  BASKETS,  with  each  word  on  a separate  line.  That  is,  Liu 
result  should  look  like  this: 

BASKET 

BASKET 

BASKET 


BASKET 
Section  13 


You  should  find  this  an  easy  program  to  write.  But  you  need  one 
more  instruction:  you  need  to  know  how  to  m.ike  the  computer  print  on 
a new  line.  We  do  that  by  printing  a carriage  return: 

PRINT'  RETURN 

(Note  that  no  quote  marks  are  used  for  this  command.) 

Here's  a program  using  PRINT  RETURN: 

010  PRINT  'HELLO' 

013  PRINT  RETURN 
020  PRINT  'GOODBYE' 

Type  NE.  Then  type  in  this  program  and  run  it. 

The  computer  will  print: 

RUN 

HELLO 

GOODBYE 

HALT 

030 

Sec  tion  14 


Now  try  Problem  3:  Modify  your  program  from  Problem  2 (see  Sec.  9)  to 
make  the  computer  print  an  endless  vertical  string  of  BASKETS,  one  on 
each  line,  using  the  procedures  given  on  Page  3 for  modifying  your  pro- 
gram. 

After  you've  made  your  modifications,  list  your  new  program  to  see  what 
it  looks  like.  Then  run  the  program  to  lie  sure  it  works. 


Figure  2 


A Portion  of  the  Printed  Instructions 


